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An  analysis  of  alternatives  (AoA)  has  long  been  required  for  major  defense  acquisition 
programs  (MDAPs)  and  major  automated  information  system  (MAIS)  programs  as  a 
matter  of  DoD  policy  (DoDI  5000.02)  and  has  recently  become  a  requirement,  as  a 
result  of  the  2366  certification  process  and  the  Weapon  Systems  Acquisition  Reform 
Act  (WSARA)  of  2009.  While  DoD  has  a  long  history  of  conducting  AoAs  prior  to 
initiating  major  programs,  the  new  statutory  guidance  on  AoAs  has  increased  AoAs'  importance 
and  emphasis.  This  is  consistent  with  an  intentional,  concerted  transition  in  the  Department's 
acquisition  culture  to  devoting  more  attention  and  planning  to  major  programs  earlier  in  the 


Husband  is  a  professor  of  cost  analysis  at  DAU  with  78  years  of  acquisition  experience  in  cost  estimating,  systems  engineering,  and  R&D 
project  management  Kaspersen  is  an  operations  research  analyst  for  Cost  Assessment  and  Program  Evaluation  (CAPE)  in  the  Office  of  the 
Secretary  of  Defense,  with  17  years  of  experience  supporting  the  acquisition  process  in  a  variety  of  analyticai  capacities. 


1 


Defense  AT&L:  May-June  2012 


process  (i.e.,  prior  to  milestone  B). 

The  statutory  references  to  AoAs 
are  brief  and  could  be  con¬ 
strued  as  inconsequential— 
merely  codifying  what  had 
previously  been  required 
under  DoD  policy.  How¬ 
ever,  the  effects  of  the 
statutory  guidance  on 
AoAs  have  been  more 
significant  than  a  sim¬ 
ple  reading  of  the  law 
might  indicate. 

Why  Do  an  AoA? 

One  answer  is  obvious:  It 
is  required  for  ACAT1 A  and 
1AM  programs.  However,  the 
more  salient  reason,  not  only  for 
MDAPs  and  MAIS  programs,  but 
also  for  smaller  programs,  is  that  an 
AoA  positions  the  Department  to  pursue 
the  most  effective  course  to  obtaining  the  capabil¬ 
ity.  An  AoA  is  simply  a  structured,  unbiased  framework  for 
evaluating  the  most  promising  alternatives  for  a  given  need, 
comparing  the  merits  and  risks  of  each,  and  providing  a  recom¬ 
mended  approach  to  decision  makers.  Clearly,  it  makes  sense 
for  large  programs  to  devote  considerable  time  and  effort  to 
an  AoA  prior  to  investing  significant  resources  in  a  specific 
approach.  But  ideally,  AoAs  should  be  done  for  any  DoD  effort 
that  has  multiple  alternatives,  just  as  individuals  perform  an 
alternatives  analysis  when  facing  a  major  personal  decision. 
Whether  we  realize  it  or  not,  most  of  us  probably  do  a  mini- 
AoA,  at  least  in  our  minds,  for  personal  decisions  like  selecting 
a  university,  choosing  between  job  offers,  or  whether  to  buy 
a  new  or  used  car.  Just  as  we  shouldn't  pull  the  trigger  on  a 
personal  decision  based  on  the  advice  of  one  vendor,  the  De¬ 
partment  shouldn't  engage  in  expensive  technology  develop¬ 
ment  without  considering  various  alternatives  in  a  systematic 
fashion.  Thought  about  this  way,  an  AoA  is  a  prerequisite  to 
initiating  a  sound  acquisition  program. 

Recent  Statutory  Guidance 

For  MAIS  programs,  the  statutory  guidance  to  conduct  an  AoA 
prior  to  program  initiation  is  contained  in  Section  811  of  the 
2001  National  Defense  Authorization  Act  (NDAA).  Subtitle 
III  of  title  40  of  U.S.  Code  (also  known  as  the  Clinger-Cohen 
Act  of  1996)  also  addresses  the  requirement  to  conduct  an 
AoA  and  economic  analysis  for  all  information  technology  in¬ 
vestments.  For  MDAPs,  Section  2366b  of  title  10,  U.S.  Code, 
added  by  the  NDAA  of  2006,  requires  that  the  Milestone 
Decision  Authority  (MDA)  certify  to  Congress  that  DoD  has 
conducted  an  AoA  prior  to  milestone  B  approval;  in  2009, 
WSARA  added  a  similar  provision  to  Section  2366a  of  title 
10  that  requires  an  AoA  be  conducted  prior  to  milestone  A. 
WSARA  also  specifically  directed  that  the  director  of  Cost 
Assessment  and  Program  Evaluation  (DCAPE)  "take  the  lead 


in  the  development  of  study  guidance  for  an 
analysis  of  alternatives  for  each  joint  mil¬ 
itary  requirement  for  which  the  chair¬ 
man  of  the  JROC  is  the  validation 
authority."  However,  in  addition  to 
assigning  responsibility  for  study 
guidance,  WSARA  further  stip¬ 
ulated  that  the  DCAPE  "shall 
serve  as  the  principal  official 
within  the  senior  management 
of  DoD  for  the ...  performance 
of  such  analyses."  These  few 
sentences  in  WSARA  have 
f  had  a  significant  impact  on 
how  AoAs  are  conducted  in  the 
Department. 

CAPE’S  Expanded  Role 

Prior  to  WSARA,  the  director  of  pro¬ 
gram  analysis  &  evaluation  (PA&E), 
cape's  predecessor  organization,  also  was 
responsible  for  drafting  study  guidance  for  AoAs 
and  for  conducting  sufficiency  reviews  documenting  how 
well  the  AoA  met  that  guidance.  However,  the  guidance  and 
review  prepared  by  PA&E  were  provided  to  the  USD(AT&L)  as 
internal  DoD  documents;  the  director  of  PA&E  did  not  have  the 
authority  or  responsibility  to  reject  an  AoA  or  direct  that  it  be 
redone.  To  be  sure,  the  D,  PA&E's  advisory  powers  were  sig¬ 
nificant;  however,  he  was  not  the  decision  authority  for  AoAs. 
The  elevation  of  the  DCAPE  to  a  Senate-confirmed  position 
changed  the  dynamic  by  giving  the  DCAPE  responsibility  not 
just  for  formulating  the  study  guidance,  but  also  for  the  per¬ 
formance  of  the  analysis.  This  responsibility  no  longer  is  just 
advisory,  since  the  DCAPE  can  be  called  to  testify  before  Con¬ 
gress  on  any  matter  within  her  portfolio.  It  naturally  follows 
that  CAPE  senior  officials  and  analysts  now  have  a  more  active 
role  throughout  the  AoA  execution  process,  rather  than  just 
during  formulation  of  the  study  guidance  and  again  at  the  end 
of  the  process  during  the  sufficiency  review  stage. 

Prior  to  WSARA,  there  was  no  statutory  guidance  dictat¬ 
ing  how  an  AoA  should  be  conducted  or  the  minimal  con¬ 
tent  it  must  contain.  WSARA  specifies  that  the  DCAPE 
must  ensure  that  the  AoA  study  guidance  requires  both 

•  "Full  consideration  of  possible  trade-offs  among  cost, 
schedule,  and  performance  objectives  for  each  alternative 
considered"  and 

•  "An  assessment  of  whether  or  not  the  joint  military  re¬ 
quirement  can  be  met  in  a  manner  that  is  consistent  with 
the  cost  and  schedule  objectives  recommended  by  the 
Joint  Requirements  Oversight  Council." 

Thus,  at  a  minimum,  CAPE  is  charged  with  assessing  how 
well  the  AoA  conducts  tradeoffs  between  alternatives  and 
whether  those  alternatives  can  be  realized  in  accordance  with 
cost  and  schedule  objectives.  Proper  consideration  of  trad- 
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eoffs  includes  ensuring  at  least  two  things:  (1)  that  a  range  of 
sufficiently  different  alternatives  are  examined  and  (2)  that 
the  alternatives  themselves  have  been  adequately  evaluated. 
What  determines  whether  an  alternative  has  been  adequately 
evaluated?  No  simple  answer  suffices  for  all  situations,  but 
a  real  example  may  be  useful.  If  AoA  results  indicate  that  a 
subsystem  specification— say  gun  caliber  or  engine  power— is 
key  to  mission  performance,  the  analysis  would  be  less  than 
adequate  if  only  one  of  the  alternatives  considered  included 
that  minimal  subsystem  specification. 

Consider  Cost  and  Affordability  Upfront 

One  outcome  of  greater  emphasis  on  upfront  planning  has 
been  an  enhanced  expectation  that  cost  and  affordability  of 
potential  DoD  systems  are  considered  earlier  in  the  process, 
including  much  more  intensively  during  requirements  genera¬ 
tion  and  at  early  major  milestones,  such  as  materiel  develop¬ 
ment  decision  and  milestone  A.  This  is  particularly  evident 
in  the  Better  Buying  Power  (BBP)  Initiatives  championed  by 
Deputy  Secretary  Ashton  Carter  and  Acting  Under  Secretary 
Frank  Kendall,  in  which  the  first  of  the  five  Thrust  Areas  con¬ 
cern  "Targeting  Affordability  and  Controlling  Cost  Growth."  In 
fact,  the  first  initiative,  "Mandating  Affordability  as  a  Require¬ 
ment,"  stipulates  that  an  affordability  target  be  established  at 
milestone  A  and  not  changed  without  the  MDA's  concurrence. 
This  is  a  major  change  from  the  way  the  Department  operated 
only  a  few  years  ago. 

As  a  cost  analyst  in  the  OSD/CAIG  in  the  mid-2000s,  one 
of  the  authors  (Husband)  saw  the  shift  in  emphasis  by 
DoD  senior  leaders  to  attaining  earlier  information  about 
programs,  including  the  desire  for  an  independent  cost  es¬ 
timate  (ICE)  at  milestone  A.  This  was  seldom  done  at  the 
time  but  became  statutory  with  WSARA  in  2009. 1  remem¬ 
ber  thinking  it  wasn't  going  to  pay  off;  it  seemed  to  me  that 
senior  leaders  didn't  have  enough  time  to  review  MDAPs 
at  milestones  B  and  C,  so  how  were  they  going  to  spend 
time  on  programs  at  milestone  A  and  earlier?  I  was  also 
skeptical  about  the  value  of  the  information  available  at  the 
early  stages.  A  cost  estimate  at  milestone  A  is  much  more 
uncertain  and  challenging  than  one  done  at  milestone  B  or 
C,  because  of  the  ill-defined  nature  of  the  program  at  such 
an  early  stage  and  the  number  of  different  solutions  that 
must  be  costed.  However,  I  believe  DoD's  culture  has  in¬ 
deed  shifted  in  recent  years  as  a  result  of  senior  leadership's 
increased  emphasis  on  upfront  planning,  and  I  also  believe 
I  was  wrong  about  the  lack  of  payoff.  Three  examples  of 
senior  leader  engagement  at  milestone  A  or  earlier  are  cited 
in  Deputy  Secretary  Carter's  Sept.  14, 2010  guidance  memo 
(Ohio-class  submarine,  presidential  helicopter,  and  ground 
combat  vehicle).  This  senior-leader  involvement  has  had  a 
significant  impact,  resulting  in  more-affordable,  less-risky 
acquisition  strategies,  based  on  more  mature  technologies, 
than  originally  envisioned. 

Other  examples  of  early  scrutiny  of  affordability  include 
portfolio  reviews,  such  as  those  for  conventional  weapons. 


ground  vehicles,  unmanned  aircraft,  and  ground-moving 
target  indicator  development.  Such  reviews  are  challenging 
because  they  require  that  better,  higher-fidelity  information 
be  provided  earlier— not  just  from  cost  estimators,  but  from 
all  disciplines.  However,  if  such  reviews  lead  to  a  better  un¬ 
derstanding  of  the  costs  associated  with  capabilities  identi¬ 
fied  in  our  requirements  documents,  the  Department  will 
benefit  from  the  time  spent  producing  better  information 
earlier  in  the  process. 

This  enhanced  emphasis  on  affordability  is  also  evidenced 
in  discussions  that  occur  in  Functional  Capability  Board  and 
Joint  Capability  Board  meetings  prior  to  Joint  Requirements 
Oversight  Council  reviews.  Senior  leaders  now  expect  the  Ser¬ 
vices  to  be  able  to  articulate  how  and  why  their  recommended 
alternative  is  affordable,  especially  when  the  recommended 
option  has  a  higher  life  cycle  cost  than  lower-performing  alter¬ 
natives.  It  is  no  longer  satisfactory  to  assert  that  a  system  is 
affordable  simply  because  the  Service  is  willing  to  fully  fund  it; 
decisionmakers  want  to  see  and  understand  the  analysis  com¬ 
paring  the  effectiveness  and  costs  of  the  alternatives  before 
they  approve  a  given  approach.  This  is  the  primary  purpose  of 
an  AoA— to  fully  evaluate  and  fairly  compare  the  effectiveness 
and  cost  of  all  the  viable  alternatives. 

Prior  to  WSARA,  a  criticism  often  heard  about  the  DoD's  AoA 
process  was  that  the  Service  had  already  selected  its  preferred 
alternative  prior  to  the  AoA  and,  as  a  result,  the  AoA  was  just  a 
rubber-stamp  document  to  "fill-the-square"  and  move  forward 
with  the  acquisition  process.  A  corollary  criticism  was  that  the 
AoA  was  often  "outsourced"— i.e.,  conducted  by  an  organiza¬ 
tion  outside  the  Service,  and  as  a  result,  the  Service  wasn't 
involved  enough  in  the  execution  of  the  AoA.  So,  a  measure 
of  improvement  for  AoAs  conducted  under  the  new,  WSAR  A- 
mandated  process  is  how  well  they  compare  the  effectiveness 
and  affordability  of  the  options.  AoAs  that  do  that  and  that  also 
effectively  defend  the  case  for  the  recommended  alternative 
will  be  valued  by  DoD  decisionmakers  and,  most  importantly, 
will  provide  a  strong  foundation  for  a  successful  acquisition 
program. 

Improving  Interaction  During  AoA  Execution 

Since  WSARA  passed  in  2009,  only  a  few  AoAs  have  been 
through  the  complete  process,  from  promulgation  of  study 
guidance  to  AoA  out-brief.  There  is  no  DoD  regulation  or  in¬ 
struction  that  guides  the  AoA  process.  As  a  result,  the  Services 
have  latitude  in  how  they  execute  an  AoA,  as  long  as  they 
accomplish  the  objectives  in  the  study  guidance.  However,  a 
few  best  practices  naturally  arise.  To  ensure  good  interaction 
and  communication  between  the  Service  and  OSD  stakehold¬ 
ers,  an  SES-level  study  advisory  group  (SAG)  is  established  to 
monitor  results  during  AoA  execution.  This  group  is  chaired  by 
CAPE  and  includes  AT&L,  Joint  staff,  and  Service  representa¬ 
tives.  Correspondingly,  the  Service  may  establish  a  Service  Ex¬ 
ecutive  Steering  Board  (SESB),  an  SES-level  body  that  includes 
Service  stakeholders  from  the  requirements,  acquisition,  and 
resource  communities. 
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The  purpose  of  both  the  SESB  and  SAG  is 
to  ensure  that  senior  leaders  are  famil¬ 
iar  with  emerging  AoA  results  and 
can  guide  or  redirect  the  effort 
as  necessary,  rather  than  being 
surprised  by  results  at  the 
conclusion  of  the  AoA.  Es¬ 
tablishing  a  SESB  also  dem¬ 
onstrates  that  the  Service 
is  invested  in  the  process, 
as  opposed  to  "outsourc¬ 
ing"  the  AoA  or  assigning 
it  solely  to  a  Service  analyt¬ 
ical  organization.  The  SAG 
and  SESB  most  likely  will 
be  linked  through  common 
participants,  which  improves 
communication  throughout 
the  process  and  lowers  the  likeli¬ 
hood  that  surprising  results  or  unex¬ 
pected  objections  will  be  encountered. 

These  groups  also  help  facilitate  another 
best  practice— that  of  "transparent  analysis.' 

Prior  to  WSAR  A,  after  the  AoA  study  guidance  was  released  to 
the  Service,  there  was  often  little  or  no  interaction  with  PA&E 
or  AT&L  until  the  AoA  report  and  briefing  had  been  completed 
and  fully  coordinated  through  the  Service.  In  an  actual  extreme 
case,  a  Component  performing  an  AoA  provided  no  updates 
during  execution  and  provided  the  read-ahead  literally  on  the 
morning  of  the  briefing  to  PA&E  leadership.  Now,  to  be  suc¬ 
cessful,  more  interaction  with  CAPE  and  AT&L  during  AoA 
execution  is  desired  and  expected.  That  doesn't  mean  that 
OSD  personnel  should  direct  the  study  or  have  significant 
influence  on  AoA  execution  decisions— there  may  be  times 
when  Service-only  meetings  are  necessary.  But  keeping  OSD 
analysts  informed  and  involved  to  the  extent  they  are  comfort¬ 
able  will  pay  dividends  in  the  long  run,  as  it  enables  them  to 
inform  their  leadership,  so  there  are  no  big  surprises  when  the 
AoA  results  are  reported  and  reviewed. 

AoA;  Where  Requirements,  Acquisition  and 
Resourcing  Communities  Come  Together 

As  much  as  analysis  has  been  mentioned,  one  might  suppose 
that  AoAs  are  prepared  solely  by  analysts.  In  fact,  that  should 
not  be  the  case.  A  wide-range  of  organizations  should  be  rep¬ 
resented  on  the  study  team  executing  an  AoA,  not  only  from 
the  requirements,  acquisition,  and  resourcing  disciplines,  but 
also  from  functional  areas  such  as  intelligence,  logistics,  tech¬ 
nologists,  testers,  operators,  etc.  One  should  also  consider 
involving  contractors  and  non-DoD  organizations;  contrac¬ 
tor  input  (properly  safeguarded)  can  be  useful  in  defining  the 
feasibility  of  various  alternatives,  while  non-DoD  organizations 
sometimes  have  key  information  or  a  shared  set  of  require¬ 
ments  that  impact  DoD  programs.  Deficiencies  in  past  AoAs 
have  sometimes  resulted  from  lack  of  interaction  between 
the  requirements  developers  (or  users)  and  the  acquisition 


community  that  conducted  the  AoA.  If  the 
requirements  are  just  "thrown  over  the 
fence"  to  the  acquisition  community 
and  there  is  little  interaction  dur¬ 
ing  AoA  execution,  it  is  difficult 
to  make  affordability  trades, 
for  instance. 

What  should  an  analyst  or 
acquisition  professional  do 
about  requirements  that 
are  deemed  unaffordable? 
Ideally,  user  input  would  be 
solicited  to  modify,  tailor,  or 
prioritize  the  requirements. 
But  if  the  user  isn't  involved, 
those  performing  the  AoA  are 
likely  to  make  the  best  judgment 
they  can  based  on  the  information 
available.  Similarly,  if  an  AoA  is  con¬ 
ducted  without  adequately  defined  re¬ 
quirements  and  the  user  isn't  involved,  ana¬ 
lysts  or  acquisition  personnel  are  forced  to  make 
judgments  about  requirements  and  capability  gaps  that  the 
user  should  establish.  Those  conducting  the  AoA  will  undoubt¬ 
edly  put  their  best  thought  and  effort  into  the  problem,  but  the 
AoA  is  likely  to  suffer  from  the  lack  of  user  involvement.  The 
AoA,  more  than  any  other  acquisition  document,  is  where  the 
requirements,  acquisition,  and  resourcing  communities  come 
together  to  decide  which  affordable  alternative  is  best. 

AoA;  The  Foundation  of  a  Successful 
Acquisition  Program 

A  well-executed  AoA  doesn't  guarantee  program  success; 
many  more  things  must  be  done  right  to  effectively  deliver  ca¬ 
pability.  It's  clear,  however,  that  a  poorly  executed  AoA  makes 
success  more  difficult.  Analysis  that  doesn't  fully  examine  the 
tradespace  or  that  is  biased  toward  one  alternative  may  lead 
to  pursuit  of  an  infeasible  or  inferior  option.  At  the  very  least,  it 
establishes  a  precedent  that  incomplete  or  insufficient  analy¬ 
sis  is  acceptable,  which  is  likely  to  become  a  pattern  as  the 
program  proceeds.  In  the  current  fiscal  environment,  in  which 
the  DoD  has  rightly  been  criticized  for  spending  as  much  as 
40  percent  of  its  acquisition  funding  on  programs  that  were 
terminated  before  delivering  capability,  it  is  more  important 
than  ever  to  start  a  program  on  a  firm  foundation  with  a  well- 
executed  AoA. 

When  deciding  where  to  live  for  the  next  20  years,  a  prudent 
individual  will  consider  all  the  viable  alternatives— buying  new, 
buying  an  existing  home,  custom  building,  renting,  perhaps 
even  living  on  a  houseboat;  the  same  principle  applies  to  DoD 
before  it  invests  in  a  weapon  system  it  will  likely  use  for  20 
years  or  more. 

The  authors  can  be  reached  at  mark. husbandcSdau. mil  and  keith. 
kasperseiKSosd.mil. 
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